This is essentially by definition--intervention on Z doesn't change the potential outcomes, so it doesn't change the value of f (X). If f is a counterfactually invariant predictor: 1. Let L be either square error or cross entropy loss. Suppose that the target distribution Q is causally compatible with the training distribution P . Suppose that any of the following conditions hold: 1. the data obeys the anti-causal graph 2. the data obeys the causal-direction graph, there is no confounding (but possibly selection), and the association is purely spurious, Y X | X We begin with the anti-causal case.

Informally, a `spurious correlation' is the dependence of a model on some aspect of the input data that an analyst thinks shouldn't matter. In machine learning, these have a know-it-when-you-see-it character; e.g., changing the gender of a sentence's subject changes a sentiment predictor's output. To check for spurious correlations, we can `stress test' models by perturbing irrelevant parts of input data and seeing if model predictions change. In this paper, we study stress testing using the tools of causal inference. We introduce \emph{counterfactual invariance} as a formalization of the requirement that changing irrelevant parts of the input shouldn't change model predictions. We connect counterfactual invariance to out-of-domain model performance, and provide practical schemes for learning (approximately) counterfactual invariant predictors (without access to counterfactual examples). It turns out that both the means and implications of counterfactual invariance depend fundamentally on the true underlying causal structure of the data. Distinct causal structures require distinct regularization schemes to induce counterfactual invariance. Similarly, counterfactual invariance implies different domain shift guarantees depending on the underlying causal structure. This theory is supported by empirical results on text classification.